Evaluation of the quality of clinical guidelines for prophylaxis of venous thromboembolism in urological surgeries by the AGREE II review instrument

Abstract Background and Aims Venous thromboembolism (VTE) is the most common cause of death during the first 30 days after surgery. There is not any study which critically evaluated clinical guidelines related to VTE prophylaxis in urological surgeries. Therefore, in this study, we decided to evaluate related clinical guidelines using the AGREE II instrument to take a positive step towards improving the care of these patients. Methods The latest version of all available clinical guidelines related to the topic of VTE prophylaxis in urological surgeries until 2021 was searched. Four appraisers, including one urologist, one cardiologist, one epidemiologist, and one MD who had prior knowledge of working with the AGREE II tool and international articles in this field appraised selected clinical guidelines. Using the AGREE II review tool, clinical guidelines were critically evaluated. Then, the score of six domains of AGREE II for each guideline was calculated and compared with each other, and the relationship between the domains was measured by Kendall's correlation test. To determine the reliability of the test, interclass correlation coefficients were calculated for all indicators. Results Items were rated on a 7‐point scale from 1 (strongly disagree) to 7 (strongly agree). NICE, CHEST, and EAU guidelines obtained the highest scores from the Overall Assessment criteria by scoring 6, 5.75, and 5.25, respectively. There was only a correlation between the score of Overall Assessment criterion with “Applicability” domain, with Kendall's correlation coefficient of 0.867 and p = 0.015. The domains of “Clarity and presentation” and “Scope and purpose” obtained the highest standardized scores by getting 84.49% and 75.69%, respectively, and “Applicability” with 30.04% obtained the lowest standardized score. Conclusion In this study, NICE, CHEST, and EAU guidelines are suggested as clinical guidelines by obtaining the highest scores from Overall Assessment criterion.


| INTRODUCTION
The goal of evidence-based medicine is to encourage medical doctors to make rational decisions based on the best and most up-to-date evidence. In the meantime, clinical practice guidelines (CPG) play the main role by providing clinical decision-making choices, according to scientific evidence and current facilities. 1 The available evidence is sometimes limited and sometimes contradictory, and on the other hand, the facilities of clinical centers are also different in different regions. Therefore, transferring research results to the clinical field and providing practical solutions is not an easy task. 2 We need more than simple drug prescriptions to make decisions and judgments; Such as profit and loss, patient's wishes, economic problems, etc.
Therefore, to create practical solutions, it is necessary to discuss and spend lots of energy between all the people related to that particular issue. 3 In fact, CPGs are strategies that have been systematically developed and help doctors to make decisions about a specific issue. 4 In recent years, the creation of various clinical guidelines for different topics has grown a lot. 5 So that currently these CPGs exist in almost all aspects of clinical practice and health policy. In fact, the main goal of these clinical guidelines is to increase the practical use of the results obtained from the research. 6 Due to the large number of clinical guides in different subjects, medical doctors and researchers who intend to use these clinical guides will be lost in choosing a good clinical guideline. 5 On the other hand, venous thromboembolism (VTE), which consists of two components, deep vein thrombosis and pulmonary embolism, is a life-threatening complication after surgeries and the most common cause of death during the first 30 days after surgeries. 7 Considering that the risk of VTE occurrence can be reduced through medical and clinical prophylaxis, 8 the way of management and the importance of using prophylaxis of VTE in urological surgeries are noticable.
Evidence that have examined VTE prophylaxis in urological surgeries are limited 9 and so far no study has critically evaluated clinical guidelines related to VTE prophylaxis in urological surgeries. Therefore, in this study, we decided to examine and evaluate related clinical guidelines using the AGREE II instrument to take a positive step towards improving the care of these patients. We included guidelines with an exclusive or predominant focus on this topic and the decision to include or exclude a guideline was made by two authors independently. Any disagreements were resolved by consensus.

| Appraisal instrument
The AGREE II instrument consists of 23 items categorized in six domains: 1. Scope and purpose: three items that address the overall aim of the guideline, the clinical question and the target population.
2. Stakeholder involvement: four items addressing the composition, expertize and representation of the development group.
3. Rigor of development: seven items that evaluate the process of locating and synthesizing the evidence, and formulating and updating the recommendations. 4. Clarity and presentation: four items that address language and format.

5.
Applicability: three items that address the potential organizational, behavioral and cost implications of applying the guidelines. 6. Editorial independence: two items that focus on potential conflicts of interest.
Items are rated on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree). The final item of the instrument asks the appraiser to consider the overall quality of the clinical guideline and decide whether they recommend the guideline for practice.

| Quality assessment
Each clinical guideline was assessed by four appraisers; all of them used the English version of the AGREE II. The clinical guidelines were appraised and scored independently by the appraisers.
The Persian version of the AGREE II tool has been validated and shown to provide scores that are comparable with the original tool. 10

| Data analysis
Domain scores of each clinical guideline were standardized following the method recommended by the AGREE II 11 and compared among the guidelines. The intraclass correlation coefficients (ICCs) were compared among different numbers of appraisers. The ICCs above 0.75 were considered to represent good, 0.40-0.75 moderate and below 0.40 poor reliability. 12,13 The Kendall's tau correlation analysis was done to assess the correlation between the domain scores and the overall assessment. Analysis of variance tests were used to compare the overall assessment scores among the guidelines. We hypothesized that the number of algorithms in the guidelines was an important determinant of the overall assessment score and used Kendall correlation tests to assess this hypothesis.

| Characteristics of clinical guidelines
We included six clinical guidelines in our study (Box 1). No French or Persian guidelines were added and the publication year of the guidelines was from 2009 to 2021. In fact, Canadian Urological Association Guideline is an adaption of EAU guideline.

| AGREE scores
The mean standardized scores of the guidelines' domains and items have been shown in Table 1. Items were rated on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree). "Clarity and presentation" and "Scope and purpose" achieved the highest scores among the AGREE domains with the mean standardized scores of 84.49% and 75.69%, respectively. "Applicability" got the lowest score with the mean of 30.04%. We observed statistically significant differences between the domain score of "Rigor of development" with "Scope and purpose" (p = 0.022), "Clarity and presentation" (p = 0.032) and "Applicability" (p = 0.039). There was also statistically significant difference between the domain score of "Stakeholder involvement" and "Applicability" (p = 0.039). No statistically significant difference among other domain scores was found.
The overall assessment scores were compared between clinical guidelines and NICE, CHEST, and EAU obtained the highest scores ( Figure 1). Table 2 shows the correlations between the overall assessment and the domain scores. Only domain of "Applicability" (p = 0.015) had a statistically significant correlation with the overall assessment score (Kendall's tau = 0.867).
We proposed the hypothesis that the "easy identification of key recommendations and application tools" may play a role in the appraisers' perspective towards a guideline but there was no significant relationship between the overall assessment scores and the total numbers of algorithms, tables, and figures in guidelines (Kendall's tau = −0.067, p = 0.851) (Figure 2).

| AGREE validity and reliability assessment
The ICCs for each AGREE domain are shown in Table 2. "Applicability" had the highest reliability score (0.837), followed by "Scope and purpose" (0.830) and "Rigor of development" (0.814). Table 2 demonstrates that no domain's ICC reached 0.75, if a single or two appraisers performed the evaluation. If three appraisers were involved, in two domains ("Applicability" and "Scope and purpose") to four did not improve the ICC values of "Clarity and presentation" to the predefined value of 0.75 ( Figure 3).

| DISCUSSION
Various studies have applied the AGREE instrument for assessing the quality of guidelines for a specific subject. To our knowledge, this is the first systematic evaluation of the quality of clinical guidelines for the management of prophylaxis of VTE in urological surgeries.
In our study NICE, CHEST, and EAU clinical guidelines obtained higher overall assessment scores in the appraisal process and were recommended for use in practice. Among these three clinical guidelines, the NICE guideline obtained the highest standardized score in three domains of "Scope and purpose," "Rigor of development," and "Applicability" and the CHEST guideline obtained the highest standardized score in domain of "Editorial independence." Also the CHEST and the NICE guidelines obtained the highest standardized score in domain of "Clarity and presentation" together. But the EAU guideline, despite the third ranking in term of Overall assessment score, did not achieve the highest standardized score in any of six domains. T A B L E 2 Intraclass correlation coefficients for each AGREE instrument domain as a function of the number of raters and correlation coefficients between overall assessment scores and each AGREE domain score. F I G U R E 2 Correlation between the overall assessment scores and the total number of algorithms, tables, and figures in the guideline. In our study NICE, CHEST, and EAU clinical guidelines obtained higher overall assessment scores in the appraisal process and were recommended for use in practice. Although, the EAU guideline has the third ranking in term of Overall assessment score, but this guideline is urology procedure-specific. In considering these factors, we felt it would be appropriate to use by urologists.

CONFLICTS OF INTEREST STATEMENT
Behnam Shakiba is an Editorial Board member of Health Science Reports and coauthor of this article. He was excluded from editorial decisionmaking related to the acceptance of this article for publication in the journal. The remaining authors declare no conflict of interest.

TRANSPARENCY STATEMENT
The lead author Nasim Torabi affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

DATA AVAILABILITY STATEMENT
The authors confirm that the data supporting the findings of this study are available within the article.